Facial Muscle Activations from Motion Capture

Biomechanically accurate finite element models of facial musculature offer a superior accuracy in reproducing facial expressions, the ability to adapt the simulation model to a particular subject as well as a compact and biophysically meaningful parameterization of expressions in terms of the muscle activations and bone motion that give rise to them. We employ such a finite element simulation model to determine the muscle activations and kinematic configuration of the rigid bones associated with an expression from a sparse sampling of the deformation of the face surface over time, acquired using a motion capture system [1]. We use a hyperelastic, transversely isotropic constitutive model to simulate passive flesh and 32 dominant muscles of the face. Our simulation model, consisting of 840K tetrahedral elements, was created through non-rigid registration of a muscle geometry template derived from the Visible Human dataset to MRI volumetric data acquired from the motion capture subject. Using a dynamic simulation for our analysis is hindered by the computational cost associated with time integration of such complex models and the dependence of the shape on deformation history, aside from muscle action. Therefore, we use a quasistatic formulation where the model is assumed to be in force equilibrium at any point in time. This assumption is formalized in the force equilibrium equation f(X,a,b) = 0, which implicitly defines the equilibrium shape X of the deformable face model as a time-independent function of the muscle activations a and rigid bone kinematics b. Fast and robust nonlinear solvers [2] are used to determine the quasistatic shape X(a,b) associated with a given set of muscle activations and bone kinematics. Furthermore, a differentiation of the force equilibrium equation reveals that the shape Jacobians ∂X/∂a, ∂X/∂b can also be computed as the solutions to sparse, symmetric and positive definite linear systems, which can be efficiently obtained using a conjugate gradients solver. Knowledge of these differential quantities enables the use of traditional optimization schemes to match a target geometry input. Our system employs a Gauss-Newton nonlinear leastsquares optimization scheme to fit the simulated face model to motion capture input by minimizing the distance of the motion capture markers to corresponding landmarks embedded on the surface of the simulation model. Notably, the system is robust against noise or outliers in the motion capture input since the captured marker motion is effectively projected to the space of physically attainable expressions through our optimization EMBEDDED MUSCLE STRUCTURE (TOP LEFT) AND SIMULATION MESH (TOP RIGHT). DEFORMABLE MODEL FIT TO MOTION CAPTURE INPUT (BOTTOM). SIMULATED (CAPTURED) MARKERS ARE GREEN (RED).